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Problems in the Design and Interpretation 
of^Jlesearcl^jai^Jbim^^Relations^^Trainini* 

by 

Roger Harrison 

The measurement of processes of learning and the evaluation of the outcomes of the 
teaching- learning process have always posed major problems in research design. This is 
especially true where the desired outcomes are broadly defined as changes in interpersonal 
behavior: A complex of knowledge, values, perceptions, and behavioral skills. Until 

recently, behavioral scientists have tended to avoid research on human relations training 
in favor of more "researchable" topics which lend themselves to rigorous experimental design. 
In addition to or because of the difficulties in constructing respectable designs, there have 
been relatively few available publication outlets for such research, and this has further 
restricted the number of studies which have come to light. 

Recently, however, there has been a substantial increase in published research on human 
relations training, and there appears to be increased interest among behavioral scientists in 
conducting studies in this area. This paper is written in part to provide these investigators 
with a review of the problems which should be thought through when planning and conducting 
such studies. 

Unlike much research in the behavioral sciences which is primarily intelligible and of 
interest only to other researchers, research on human relations training is frequently written 
for and read by administrators and practitioners who may base important action decisions on the 
findings of the research. This latter group has an important need for guidelines which will 
aid them in evaluating the soundness and applicability of research reports to their practical 
decisions on the planning and conduct of training. This paper is equally addressed, then, to 
the creators and consumers of research on human relations training. 

Much of the research in this area is concerned with the special variation of training 
known as "sensitivity training," T-Group training, or laboratory training in human relations. 
My own experience and familiarity with the literature are also focused on this Approach, and 
the examples and references in this paper will be drawn from the literature on laboratory 
training. An attempt will be made to review thoroughly the problems of method in this area. 

These problems, however, are general to research on human relations training, not 
specific to sensitivity training as a special method. The discussion should be of utility to 
those interested in conducting or interpreting research on any of the various techniques of 
human relations education. 

In this paper I shall try to suggest a number of problems which I *eel have some currency 
in research on human relations training. I do not wish to discourage re,-, ear eh on training or 
to suggest that we postpone investigation until the ambiguities inherent In this area can be 
resolved by clever research designs or ingenious statistical manipulation. On the contrary, 
my aim is to point out some ways of avoiding difficulty as well as to indicate some difficul- 
ties which we cannot yet do much about other than simply to be aware of them. I hope that 
this modest catalogue may stimulate others to attack the problems to which solutions appear 
to be in the offing, but I hope that we shall have the courage to go ahead with the search, 
even in those areas where we appear doomed to live for the present with lack of rigor in 
design and with ambiguous, inconsistent, or even misleading findings. When all is said and 
done, scientific progress has never been stimulated by an unwillingness to launch investiga- 
tion in the face of uncertainty or the lack of elegant tools. 

I shall try, where possible, to indicate ways in which methodological barriers can be 
circumvented and ambiguities resolved by supplementary investigations. It is to be hoped 
that where I am unable to do this, others may find a way. 



1. A notable exception is the classic study by Fleishman (1951). This study showed that it is 
possible to construct rigorous training evaluation designs, given sufficient access to the 
trained subjects. 



The Problem of Controls 



The provision of adequate control groups for research on training is one of the most 
persistent methodological problems in this area. The fact that a person is in a control 
group biases his self-image and the perception of him by others; the fact that a person has 
participated in training inclines him and others to look for change in his behavior. 

To compound the problem, there often is administrative or self-control over the divi- 
sion of a group of otherwise equivalent members of an organization into a trained group and 
a control group. That is, there is other than random selection as to who receives the 
training and when. Usually, someone makes a decision to send to training a group which is 
judged more likely to benefit or more willing to participate than those who are not trained. 
Since there is nearly always some such selection of who is to be trained, this problem is 
an ubiquitous one. 

This problem cannot be resolved by enforced randomness of assignment to training. 
Sensitivity training programs are usually designed for participants who are at least nom- 
inally volunteers. If, for the purposes of research design, assignment to a training or 
control group is made a matter of administrative fiat, it changes the nature of the train- 
ing itself. Those who have worked in nonvoluntary laboratory training settings know that 
the participants show a much higher degree of resistance and mistrust than is the case 
where attendance is strictly voluntary. 

One method of administrative control which preserves a degree of voluntariness has 
been suggested by Massarik (1965). It involves delaying the participation of some volun- 
teers and using them as a control group in the interim. While this method is theoretically 
feasible, I have found in practice that it is generally impossible to prevent at least a 
few persons' joining the control group who have delayed their participation in the labora- 
tory ouu of ambivalence or reluctance to attend. 

There are two ways around the control group problem which appear to me to be both 
genuinely valid and practical. Both require that we study the proce, 2 of training as well 
as the outcomes. Instead of simply measuring participants before and after their passage 
through a "black box" called training, we must make some hypotheses about what it is that 
happens to the person which causes us to predict one outcome rather than another. 

The first solution to be suggested is feasible where it is possible to give comparable 

groups of participants training which differs systematically along some important training 
process dimension. For example, the dimension might be depth of personal involvement. The 
variations (from right to left) might be lectures, case studies, role playing, group- 
oriented T Groups, person-oriented T Groups, "personal growth" laboratories. One such 
study by Byrd (1966) compared participants in a T-Group laboratory with those in a 
"nongroup" situation that used personal growth activities as the basic learning setting. 

Each of the groups was seen by participants and others as having undergone a significant 
learning experience, yet it was possible to demonstrate differential outcomes consistent 
with Byrd’s predictions. 

Another example of the use of this kind of control is a study by Bunker and Knowles 

(in press) in which the effects of laboratories of different lengths were compared. Bunker 

and Knowles also compared the kinds and amounts of change occurring for participants of 
different occupational backgrounds. 

The assumption behind these designs is that if the amount and itind of training outcome 
vary systematically and predictably as functions of some input (whether the design, the 
type of participant, the behavior of training staff, or whatever), then the obtained 
changes can be viewed as "real." Because all groups being compared have been through a train- 
ing experience, the design eliminates the biasing of perception which occurs when an untrained 
control group is used. 
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A variation on this theme introduces control by prediction of training outcomes through 
independent process variables. If we can measure some important difference in the behavior 
of participant's or in the quality of their experience while in training, then this type of 
control can be achieved. 

This design is illustrated by a study by Banker in which it was found that the number 
of reported changes for participants in laboratory training was associated with ratings of 
active involvement in the T Group (Bunker, 1965). Similarly, the author found significant 
relationships between active involvement in training and change in categories of inter- 
personal perception following training (Harrison, 1966). In another study, the ratings of 
involvement predicted posttraining increases in consideration, as rated by organizational 
associates (Harrison & Oshry, 1967). This predictability of change was found even though 
there was no significant overall change in rated organizational behavior. If third vari 
able prediction had not been used, the study would have resulted in a misleading negative 
finding rather than a finding of differential change. 



Te mporal Change in Training Outcome 

One process which has been little investigated is the progressive change in outcome 
with time following training. We ordinarily give some lip service to the importance of 
longitudinal changes but few researchers have gathered longitudinal data. The importance 
of doing so is illustrated by one study the author conducted in which data were collected 
at two points following training. In this study of cognitive change, the changes were 
progressive over a surprisingly long period of time. It was predicted that changes would 
be greatest immediately after tne laboratory and would decay with time. Change was 
assessed six weeks after the conclusion of training and again after six months in order 
to measure temporal deterioration. The results were opposite to the prediction: The 

changes were positive but insignificant at the six-week mark, increasing to higher and 
statistically significant levels only after six months (Harrison, 1966) . 

I believe that there is a tendency for us to study outcomes without thinking deeply 
about the temporal process of change. What theory there is about this process suggests 
that this is unwise. For example, Schein and Bennis (1965) have elaborated a three-stage 
temporal theory of change originally conceptualized by Lewin. According to this model, 
the T-Group laboratory has an initial "unfreezing" effect. The individual is shaken up 
by dissonance and discomf irmation of his self-concept through the feedback which he 
receives from others. The unfreezing process creates, to a greater or lesser degree, a 

need for change . 

This is followed by a period of search and experimentation during which the individ- 
ual tries out new conceptualizations, experiments with new behavior, and attempts to 
gather information about the effectiveness of alternate ways of relating to others. 

This second phase covers the period during which behavioral change actually takes place. 

ig followed by refreezing of behavior due both to internal forces stemming from 
improved adaptation and/or defense and to external forces generated by the social 
environment. With refreezing, the individual’s behavior again comes to a quasi-stable 
equilibrium as close to a steady state as behavioral patterns usually come. The forces 
which produced the original imbalance have resulted in change up to the point where 
counter forces are generated within or external to the person which oppose the change 
and eventually restabilize behavior. 

This is a reasonable though not an empirically demonstrated model of the change 
process. As yet no one knows how long the phases may be expected to last nor how to 
identify, in practice, the transition from one to another. What is clear, however, is 
that if this model is correct the phase a person is in makes a great deal of difference 
in the kind of changes we should look for as training outcomes. What we usually mean 
when we talk about outcomes is the "refrozen" state, in which the individual has inte- 
grated and stabilized new patterns of cognition, perception, and behavior. 



What we may often measure is the process of change itself. During this phase we should 
expect rather different outcomes if the training has been successful. Rather than stabilized 
behavior patterns, we should expect to find such changes as higher activity levels and rates 
of interaction; greater risk taking ic attempting new behavior with others; greater vari- 
ability and inconsistency in behavior, values, and perceptions; and a higher level of 
aspiration for the quality of interpersonal relationships. These might well be accompanied 
by higher levels of anxiety and discomfort around the individual's self-concept and his 
interpersonal relationships. 

It is not immediately clear what instruments or methods would best get at these quali- 
tative aspects of behavior, but it is clear that they have not been explicitly investigated 
in recent studies of training outcome. 

Dimensions and Directions of Change 

A related problem has to do with the number of dimensions on which our design allows us 
to measure changes in participants. The number and kinds of degrees of freedom in our 
design are an implicit statement of the position we take regarding the aims and goals of 
training. In designing a study of outcome we express our point of view about the kinds 
and directions of change which we will classify as desirable outcomes. We may study only 
a narrow range of dimensions, in which case we are by implication narrowing the goals of 
training, or we may include any conceivable kind of change, in which case we are implying 
that one kind of change is as good as another. 

The classic studies by Bunker (1965), Miles (1965), and Valiquet (in press) are exam- 
ples of the latter approach. They all use a design of very wide focus. Each asked par- 
ticipants and their associates to give free responses describing any changes which took 
place in the participant's behavior during the preceding year. The obtained responses 
from a trained and a control group were then classified inductively, and the number of 
changes in each category were compared to determine the kinds of changes on which labora- 
tory participants differed significantly from controls. 

This method not only does not specify in advance the dimensions on which change is 
to be measured but it also tends to count a change as equally significant whether it is 
up or down on a given dimension. For example, it would be possible, using this method, 
to find significant differences between trained subjects and controls on both self-control 
and spontaneity , qualities of behavior which are generally considered polar opposites on 
a single dimension. 

By contrast, a study by Harrison and Oshry (1967) of changes in organizational behavior 
following laboratory training used an instrument constructed deductively from Argyris"' two- 
dimensional theory of organizational behavior (Rational-Technical Competence and Inter- 
personal Competence). Through factor analysis of the items derived from the theory, three 
dimensions were actually found and studied. Changes not measured by this instrument were, 
by implication, classified as irrelevant. 

Furthermore, only unidirectional changes were assessed. That is, if half the partici- 
pants showed increases in Rational-Technical Competence and half showed decreases of equal 
magnitude the net effect of the training was considered to be zero. 

A Classification Scheme for Training Outcomes 

These studies are examples of two of the three major categories into which research 
designs may be classified according to the kinds and directions of change for which the 
design permits assessment. Harrison and Oshry 's (1%7) study would be classified as 
normative with respect to outcome. It is restrictiv e with respect to the dimensions on 
which change was assessed and it is prescriptive regarding the direction of change con- 
sidered desirable. Much of the writing about sensitivity training is, in fact, normative 
with respect to outcomes. For example, it is usually considered good for people to become 
more democratic, bad for them to become more authoritarian. It is good for people to be 
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more open to their own and others' feelings; it is bad for them to reject or suppress their 
emotionality. 

The client's point of view is also generally normative with respect to outcomes. A 
personnel manager wants to know whether he can expect people he sends to a laboratory in 
human relations to come back more or less dominant, more or less responsive to the needs 
and feelings of others, more or less expressive of his own needs and feelings. He is not 
likely to be satisfied if he is told "it depends on the direction in which the individual 
needs to change." 



Among practitioners there appears to be a historical trend (for example, in the 
laboratories conducted at Bethel, Maine, by the National Training Laboratories^) from a 
normative approach focused on the development of democratic ideology and its expression 
in the democratic decision-making group to a concern with individual growth as the desired 
outcome. The individual growth position with respect to outcome would probably focus on 
the receiving of feedback in the T Group as the basic learning process in laboratory 
training. The process begins when the individual exposes his characteristic styles of 
relating to others in the T Group and receives feedback about the reactions of others to 
his behavior. In general, we would expect that rigidly extreme styles would tend to 
receive negative feedback at either end of most dimensions of behavior. For example, 
it is common for both over-talkative and under-participating members to be pressured by 
the rest of the group to approach the group average in their verbal activity. Similarly, 
both domineering and very dependent members are likely to receive feedback which, if 
heeded, would move them toward e more interdependent orientation. The same moderating 
influence tends to be exerted on both the cold and distant and the overly warm and per- 
sonal members. 



Carried to its extreme, the individual growth point of view would see the T Group as 
the place where sharp edges are rubbed off people. A more acceptable version of this 
point of view would be that the T Group is a place where each individual is encouraged to 
explore and express the latent and underdeveloped as. -ts of himself. From the standpoint 
of experimental design, these both come to much the thing. The Bunker, Miles, and 

Valiquet studies referred to above are examples of this approach, and it is noteworthy 
that there is not, to the author's knowledge, any study which has used this method which 
has failed to show significant results. The individual growth point of view seems to 
"fit" the laboratory training process very well. 

A third position with respect to outcome has seldom if ever been represented in 
experimental design, but it is theoretically very important. It is represented by Bennis ' 
1962 article on > the goals of laboratory training. According to this point of view, the 
objective of laboratory training is neither to teach everyone the same values and behaviors 
(normative model) nor to improve the adaptation of the individual by changes in values or 
behavior style which are tailored to his needs. Rather the objective is a general improve- 
ment in adaptive capability for all members, based on (1) improved accuracy of perception 
of the self and of one's relationships with others, (2) more complex and accurate cognitive 
mapping of the realm of interpersonal phenomena, occurring through the development of new 
concepts which permit the individual to comprehend a wider range of interpersonal phenom- 
ena, (3) increases in behavioral range and flexibility, through experimentation with 
hitherto avoided or unpracticed modes of relating to others , and (4) development of an 
interest in and a method for continued learning about interpersonal relationships and 
group phenomena or, as Bennis (1962) puts it, "learning how to learn." 

Little progress has been made in measuring the attainment of such goals. Doubtless 
the Bunker-Miles-Valiquet method gets at these changes as well as others, but only if the 
describer is himself sophisticated enough to observe and report abstract similarities 
among behaviors which may seem quite different at the concrete level of description. 



2. On June 1, 1967, the name of the National Training Laboratories was officially changed 
to NTL Institute for Applied Behavioral Science, associated with the NEA. 
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For example, if the individual has learned to take more risks in trying new interpersonal 
behavior as part of his "learning how to learn," he may have exhibited this in a wide 
variety of ways from which the observer would have to abstract the concept, "takes more 
risks." Consequently, although the highest and most desirable goal of laboratory training 
in human relations may be this development of the individual's adaptive and learning 
capacity, it is doubtful that it has yet been a significant object of study, and it 
probably will not be until a good deal of work is put into the creation of ways of 
quantifying such changes. I suggest that developing these methods should have a high 
priority in current research efforts. 



Variability in the Training Experience 

There are several problems involved in the actual process of working with subjects 
and gathering data in research on laboratory training. A major one has to do with the 
difficulty of specifying the nature of the training experience which each participant has. 

To begin with, there exists a kind of cult of originality among laboratory trainers in 
which a dominant value is the invention and proliferation of new variations in training 
design. It thus becomes practically impossible to standardize training design except 
insofar as the routine inclusion of the T-Group experience may be considered standardi- 
zation. A major difficulty here is that we do not yet have adequate enough theory about 
the effects of different elements of training design even to permit us to classify labora- 
tories according to design. We may suspect, for example, that there are differences between 
laboratories in which there is a good deal of formal assistance to the participant in 
conceptualizing his experience through lectures, discussions, readings, and so on. We do 
not, however, have clear hypotheses about the relationship between having an experience, 
having an insight about that experience, and conceptualizing or generalizing the insight 
to other situations. If I had to select one aspect of training which most needs theoret- 
ical formulation and exploratory investigation, I should choose this problem of the 
relationship between experience, conceptualizing activities, and learning outcome. 

A similar but conceptually less murky area has to do with the effects of variations in 
trainer style on participant learning. Though this area is far from adequately mapped, 
considerable exploration is going on. For example, Culbert (1966), Peters (1966), and 
Bolman J have all made recent contributions to our understanding. Hopefully, we shall soon 
have instruments which will permit us to assess trainer style as an independent variable 
and relate it to kind and extent of outcome. 

Our understanding of the effects of group composition is in a similar stage of early 
exploration. We certainly have consistent and repeated evidence that group composition has 
a significant effect on the learning of participants, and we know something about the kinds 
of composition variables which are relevant (for a review of the literature in this area 
see Harrison, 1965). 

It is a sobering thought indeed that if we take the effects of group composition seri- 
ously, we should try to assess the composition of each and every T Group we study. Probably, 
in studies using a large sample of groups, it is sufficient to randomize assignment of 
participants to groups and call the groups heterogeneous. It is important to remember, 
however, that a strong case can be made in favor of the proposition that participants learn 
better in groups which are intentionally composed to produce conflict between polarized 
subgroups (Harrison, 1965). So what we do for administrative convenience and simplicity 
of research design may not result in the best training of which we are capable. As an 
alternative, it may be worthwhile to explore the routine use of simple instruments and/or 
observations of participant behavior to compose groups for optimum learning. 



3. Lee Bolman cf Yale University is constructing an as-yet-unpublished instrument for 
assessing trainer behavior along several dimensions, to be rated by participants and by the 
trainer himself. Personal communication. 1966. 



The Timing of Data Collection 



There are two problems connected with the actual administrative process of data 
collection which deserve consideration in any research design. One has to do with timing. 
After considerable experience with the administration of instruments on the opening day of 
a laboratory, I have come to the conclusion that the anticipatory anxiety which staff and 
participants alike feel during this period significantly affects the direction and vari- 
ability of responses to many kinds of instruments. Unlike concerns which arise around 
events occurring in the process of training, this anticipatory anxiety seems largely 
irrelevant to any of the concerns of the research. The experience is ubiquitous but 
transitory, and when the training begins this anxiety is shortly replaced by more 
realistic concerns. 

My current preference is to administer instruments by mail prior to the laboratory, 
if possible. If not, I sometimes wait a day into the laboratory before collecting "pre- 
training" data. This introduces problems of interpretation, but I have decided that I 
can put up with moderate ambiguity of inference in preference to gathering data at a time 
when most participants are upset and anxious. 

Experimenter-Participant Relationships in the Laboratory Setting 

Experience has also convinced me that the effects on results of the relationship 
between experimenter and subjects in research on training are even more significant than 
we are finding them to be in the academic psychological laboratory. In the latter situ- 
ation, subjects are usually willing to put up with a lot of actual or suspected manipu- 
lation on the part of the experimenter in the interests of the scientific values which 
are shared by educated adults in our culture. In the training setting, however, we 
operate according to a norm which holds that the search for truth is a cooperative one 
which can only proceed successfully if the participants are as open with one another about 
their experiences, observations, and inferences as it is possible for them to be. An 
extremely high value is set upon the development and maintenance of trust between staff 
and participants and among participants themselves. 

In this atmosphere, the secrecy of an experimenter toward his subjects and his real 
or imagined manipulation of the latter are strongly opposed by the dominant values of the 
laboratory training culture. If he flies in the face of these values, the experimenter 
runs a very real and substantial risk of being isolated and outlawed and of becoming 
uninfluential. Furthermore, when the researcher does place himself in this countercultural 
position, a good deal of the suppressed resentment which has been generated by experiences 
with behavioral scientists in school or at work comes to the surface. The research is thus 
affected not only by the relationship between the present researcher and his subjects* but 
also by the latter's past experiences as a subject. 

The net result of all this is to produce various kinds and degrees of rebellious 
behavior on the part of some unknown proportion of the subjects. My guess is that in most 
research designs this rebellion results in increased variability and unreliability rather 
than in systematic bias. 

Both the timing problem and the experimenter-subject relationship are sources of 
falsely negative findings. The operation of these factors tends to produce increased 
error variance so that the relationships which the experimenter is seeking turn out to be 
in expected directions, but statistically insignificant. This may lead to nonpublication 
of results and/or the abandonment of a promising avenue of inquiry. I would hypothesize 
that the heightened sensitivity of laboratory participants to qualities of interpersonal 
relationships frequently combines with the experimenter's cold and standoffish style to 
produce such false negatives. Considerable care is required to overcome this source of 
error. To do so, one must usually run some danger of biased findings. This danger has 
been the conventional justification for the experimenter to follow his natural inclina- 
tions towards nonexposure and low involvement with subjects. I am suggesting a conscious 
reversal of this practice. This means being open with the subjects about one's intentions. 
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interests, and motives. It means making research data available wherever possible to help 
the participants in their attempts to learn about themselves It means making oneself per- 
sonally available to participants and showing a genuine interest in the personal growth 
and enhancement which they are seeking in the laboratory. 

In my experience, it is possible to move in this direction a considerable way without 
seriously compromising the canons of experimental design. It is probably not possible to 
avoid any compromise at all with scientific respectability. One must make a personal 
choice between his estimate of the dangers of falsely negative results and his trained-in 
distaste for "messy" designs in which the subjects know too much. It is not possible to 
avoid making the choice The respectably sanitary design runs significant risks of being 
a failure; the design which enlists the subjects as willing participants in the search 
for truth runs the risk of biasing effects. Ingenuity in design can moderate these 
effects, but it cannot eliminate them. 



Statistical Problems in Training Research 

Last, I should like to refer to some statistical problems which are endemic to research 
involving the measurement of change. These problems have been so well treated in a symposium 
on change edited by Harris (1963) that I, a nonstatistician, shall do no more than point to 
them. The chapters by Bereiter and Lord discuss in detail the difficulties in such procedures 
as measuring the relationship between initial standing on a test or variable and change on 
that variable, or assessing the relationship between change on a variable and an independent 
predictor of that change. They also explain the operation of an extremely important 
phenomenon in measures of change — that of regression toward the mean. Some understanding 
of these difficulties should be acquired before undertaking research involving measures of 
change so that the major pitfalls can be avoided or taken into account. 

I should like to reiterate , that while I believe it is prudent to be concerned about 
the problems of method which beset our enterprise I believe equally strongly that it is 
unnecessarily obsessive to be discouraged by them. We should not allow these considera- 
tions to dissuade us from conducting research. Nor should we become overly rejecting of 
the findings of others because they have not overcome all of the obstacles in the way of 
achieving certainty. The problems to which this paper has been devoted are difficulties; 
they are not disasters. In spite of them we have already accumulated knowledge through 
research which is sound enough to lead to significant improvements in practice and 
increases in understanding. These achievements should encourage us to learn to live with 
the ambiguities we cannot avoid while working to reduce those which are amenable to 
improved research design. It is in the service of this goal that the current review is 
offered. 
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